- 5.3.1 Orthogonal Vectors
- 5.3.2 Orthonormal Sets
- 5.3.3 Orthonormal Bases
- 5.3.4 Non-orthogonal Bases and Coordinates
- 5.3.5 Orthonormal Bases and Coordinates
- 5.3.6 Invariance of the Inner Product
- 5.3.7 Parseval's Identity
- 5.3.8 Orthogonal Matrices
- 5.3.9 Reflections
- 5.3.10 Rotations
- 5.3.11 Exercises
- copyleft
During our work with different bases in $\mathbb{R}^n$, surely you have noticed that using the standard basis $\left\{\mathbf{e}_1,\mathbf{e}_2,\ldots,\mathbf{e}_n\right\}$ is much easier than using other bases. This comes from two key properties possessed by the standard basis, all of the vectors are orthogonal to one another and each of them is a unit vector. Generally, bases with these two properties are the most desirable for any inner product space $V$.
Definition¶
Orthogonal Set
Let $S$ be a set of nonzero vectors in an inner product space $V$. If $\ \langle \mathbf{u},\mathbf{v} \rangle = 0$ for all $\mathbf{u},\ \mathbf{v}\in S$ with $\mathbf{u}\neq\mathbf{v}$, then $S$ is an orthogonal set of vectors. Orthogonal sets are also called mutually orthogonal sets.
Example 1 - Orthogonal Set of Vectors¶
Consider the vectors $S = \left\{\, \mathbf{v}_1,\ \mathbf{v}_2,\ \mathbf{v}_3 \,\right\}$ such that
$$ \mathbf{v}_1 = \left[ \begin{array}{r}\ 1\ \\ -1\ \\ \ 1\ \end{array} \right] \qquad\qquad \mathbf{v}_2 = \left[ \begin{array}{r}\ 1\ \\ \ 1\ \\ \ 0\ \end{array} \right] \qquad\qquad \mathbf{v}_3 = \left[ \begin{array}{r} -1\ \\ \ 1\ \\ \ 2\ \end{array} \right] $$
These form an orthogonal set since each pair of vectors is orthogonal:
$$ \begin{align*} \langle \mathbf{v}_1,\mathbf{v}_2 \rangle = \begin{bmatrix}\ 1\ & -1\ &\ 1\ \end{bmatrix}\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 0\ \end{bmatrix} &= (1)(1) + (-1)(1) + (1)(0) = 0 \\ \\ \langle \mathbf{v}_1,\mathbf{v}_3 \rangle = \begin{bmatrix}\ 1\ & -1\ &\ 1\ \end{bmatrix} \begin{bmatrix} -1\ \\ \ 1\ \\ \ 2\ \end{bmatrix} &= (1)(-1) + (-1)(1) + (1)(2) = 0 \\ \\ \langle \mathbf{v}_2,\mathbf{v}_3 \rangle = \begin{bmatrix}\ 1\ &\ 1\ &\ 0\ \end{bmatrix} \begin{bmatrix} -1\ \\ \ 1\ \\ \ 2\ \end{bmatrix} &= (1)(-1) + (1)(1) + (0)(2) = 0 \end{align*} $$
Orthogonal sets are also linearly independent, which makes them usable as a basis for the subspace $\mathrm{Span}(S)$. In this example, $S$ is a set of three linearly independent vectors in $\mathbb{R}^3$, so the set is a basis for $\mathbb{R}^3$.
Theorem 1¶
Orthogonal Sets are Linearly Independent
If $S\subset V$ is an orthogonal set of nonzero vectors in an inner product space $V$, then $S$ is a linearly independent set.
Proof of Theorem 1¶
Suppose $S$ is a collection of mutually orthogonal vectors in inner product space $V$. For any finite subcollection $U = \left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_n \,\right\} \subseteq S$ suppose a linear combination of these vectors is $\mathbf{0}$
$$ \sum_{i=1}^n c_i\mathbf{u}_i = c_1\mathbf{u}_1 + \cdots + c_n\mathbf{u}_n = \mathbf{0} $$
Computing the inner product of this linear combination with any vector $\mathbf{u_j}\in U$ yields
$$ \begin{align*} \left\langle \mathbf{u}_j,\ \sum_{i=1}^n c_i\mathbf{u}_i \right\rangle &= \left\langle \mathbf{u}_j,\mathbf{0} \right\rangle = 0 \\ \\ \sum_{i=1}^n c_i\left\langle \mathbf{u}_j,\mathbf{u}_i \right\rangle &= 0 \\ \\ c_1\langle\mathbf{u}_j,\mathbf{u}_1 \rangle + \cdots + c_n\langle \mathbf{u}_j,\mathbf{u}_n \rangle = \sum_{i=1}^n c_i\langle\mathbf{u}_j,\mathbf{u}_i\rangle &= 0 \\ \\ c_1\cdot0 + \cdots + c_j\langle \mathbf{u}_j,\mathbf{u}_j \rangle + \cdots + c_n\cdot0 = c_j\left\lVert \mathbf{u}_j \right\rVert^2 &= 0 \\ \\ c_j &= 0 \end{align*} $$
Since this is true for any choice of $j\in\left\{1,\dots,n\right\}$, all scalars $c_1=c_2=\ldots=c_n=0$. Therefore $U$ is a linearly independent set. Since this is true for any finite subcollection of vectors in $S$, $S$ is a linearly independent set. $\blacksquare$
Exercise 1 - Orthogonal Sets and Independence¶
Consider the set of vectors in $\mathbb{E}^3$
$$ S = \left\{\, \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix},\ \begin{bmatrix}\ \ 1\ \\ -1\ \\ \ \ 0\ \end{bmatrix},\ \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ -2\ \end{bmatrix} \,\right\}. $$
Show that $S$ is an orthogonal set, and explain why $S$ must be a basis for $\mathbb{E}^3$.
Check Your Answer
Each pair of distinct vectors is orthogonal: $\langle(1,1,1),(1,-1,0)\rangle = 0$, $\langle(1,1,1),(1,1,-2)\rangle = 0$, and $\langle(1,-1,0),(1,1,-2)\rangle = 0$. Since the vectors are nonzero and mutually orthogonal, they are linearly independent by Theorem 1. Three linearly independent vectors in the three dimensional space $\mathbb{E}^3$ form a basis, so $S$ is a basis for $\mathbb{E}^3$.
The set $\left\{\, \mathbf{u}_1,\ \dots,\ \mathbf{u}_n \,\right\}$ of unit vectors is orthonormal if and only if
$$ \langle \mathbf{u}_i,\mathbf{u}_j \rangle = \delta_{ij} $$
where
$$ \delta_{ij} = \left\{ \begin{matrix} 1, & \quad i= j \\ 0, & \quad i\neq j \end{matrix} \right. $$
is the Kronecker delta.
An orthogonal set of vectors $S = \left\{\, \mathbf{v}_1, \dots, \mathbf{v}_n \,\right\}$ may be transformed into an orthonormal set by dividing each vector by its norm
$$ \mathbf{u}_j = \frac{ \mathbf{v}_j }{ \lVert \mathbf{v}_j \rVert },\ 1\le j\le n $$
The new set $\left\{\mathbf{u}_1,\mathbf{u}_2,\ldots,\mathbf{u}_n\right\}$ is an orthonormal set.
Example 2 - Orthonormal Set¶
In Example 1, we showed the set $S = \left\{\, \mathbf{v}_1,\ \mathbf{v}_2,\ \mathbf{v}_3 \,\right\}$ is an orthogonal set. We can see by inspection that the vectors
$$ \mathbf{v}_1 = \begin{bmatrix}\ \ 1 \\ -1 \\ \ \ 1 \end{bmatrix} \qquad\qquad \mathbf{v}_2 = \begin{bmatrix}\ \ 1 \\ \ \ 1 \\ \ \ 0 \end{bmatrix} \qquad\qquad \mathbf{v}_3 = \begin{bmatrix} -1 \\ \ \ 1 \\ \ \ 2 \end{bmatrix} $$
are not unit vectors. Dividing each vector by its length yields
$$ \mathbf{u}_1 = \begin{bmatrix}\ \ \frac{1}{\sqrt{3}} \\ -\frac{1}{\sqrt{3}} \\ \ \ \frac{1}{\sqrt{3}} \end{bmatrix} \qquad\qquad \mathbf{u}_2 = \begin{bmatrix} \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \\ 0 \end{bmatrix} \qquad\qquad \mathbf{u}_3 = \begin{bmatrix} -\frac{1}{\sqrt{6}} \\ \ \ \frac{1}{\sqrt{6}} \\ \ \ \frac{2}{\sqrt{6}} \end{bmatrix} $$
The set $U = \left\{\, \mathbf{u}_1,\ \mathbf{u}_2,\ \mathbf{u}_3 \,\right\}$ is an orthonormal set.
Exercise 2 - Normalizing an Orthogonal Set¶
The set
$$ S = \left\{\, \begin{bmatrix}\ \ 2\ \\ \ \ 1\ \\ \ \ 2\ \end{bmatrix},\ \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -2\ \end{bmatrix},\ \begin{bmatrix}-2\ \\ \ \ 2\ \\ \ \ 1\ \end{bmatrix} \,\right\} $$
is an orthogonal set in $\mathbb{E}^3$. Find the orthonormal set obtained by normalizing $S$.
Check Your Answer
Each vector has norm $\sqrt{2^2+1^2+2^2} = 3$, so dividing each by $3$ gives the orthonormal set $$ \left\{\, \frac{1}{3}\begin{bmatrix}\ \ 2\ \\ \ \ 1\ \\ \ \ 2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}-2\ \\ \ \ 2\ \\ \ \ 1\ \end{bmatrix} \,\right\}. $$
Definition¶
Orthonormal Basis
In an inner product space $V$, if $\mathcal{B}$ is an orthonormal set, then we say that $\mathcal{B}$ is an orthonormal basis for subspace $S = \mathrm{Span}(\mathcal{B})$.
The primary benefits of using orthonormal bases are
finding the coordinates of vector $\mathbf{v}$ with respect to an orthonormal basis is simpler
projections of $\mathbf{v}$ onto subsets spanned by an orthonormal set are immediate
inner product and norm may be computed directly from the coordinates of a vector
As such, we tend to do linear algebra using orthonormal bases. The following example features a particularly useful orthonormal basis, one whose application we will discuss later in the section.
Example 3 - Canonical Basis for $\mathbb{E}^3$¶
$\left\{\,\ihat,\,\jhat,\,\khat\,\right\}$ is an orthonormal basis for Euclidean inner product space $\mathbb{E}^3$.
Example 4 - Canonical Basis for $\mathbb{E}^n$¶
The canonical basis $\left\{\,\mathbf{e}_1,\,\mathbf{e}_2,\,\dots,\,\mathbf{e}_n\,\right\}$ is an orthonormal basis for Euclidean inner product space $\mathbb{E}^n$.
Example 5 - Fourier Basis¶
Consider inner product space $C[-\pi,\pi]$ with inner product
$$ \langle f,g \rangle = \frac{1}{\pi}\int_{-\pi}^\pi f(x)g(x)\, dx.$$
Let $\mathcal{F}_n = \left\{\, \frac{1}{\sqrt{2}},\ \cos x,\ \sin x,\ \cos 2x,\ \sin 2x,\ \ldots,\ \cos nx,\ \sin nx \,\right\}$. This set and its span are called trigonometric polynomials. Set $\mathcal{F}_n$ is orthogonal since for any positive integers $j$ and $k$
$$ \begin{align*} \left\langle \frac{1}{\sqrt{2}},\cos jx \right\rangle &= \frac{1}{\pi\sqrt{2}}\int_{-\pi}^\pi \cos jx\, dx = 0 \\ \\ \left\langle \frac{1}{\sqrt{2}},\sin kx \right\rangle &= \frac{1}{\pi\sqrt{2}}\int_{-\pi}^\pi \sin kx\, dx = 0 \\ \\ \langle \cos jx,\cos kx \rangle &= \frac{1}{\pi}\int_{-\pi}^\pi \cos jx \cos kx\, dx = 0 \qquad j\neq k \\ \\ \langle \cos jx,\sin kx \rangle &= \frac{1}{\pi}\int_{-\pi}^\pi \cos jx \sin kx\, dx = 0 \\ \\ \langle \sin jx,\sin kx \rangle &= \frac{1}{\pi}\int_{-\pi}^\pi \sin jx \sin kx\, dx = 0 \qquad j\neq k \end{align*} $$
The $\cos jx$ and $\sin kx$ terms for positive integers $j$ and $k$ are already unit vectors because
$$ \begin{align*} \langle \cos jx,\cos jx \rangle &= \frac{1}{\pi}\int_{-\pi}^\pi \cos^2 jx \, dx = 1 \\ \\ \langle \sin kx,\sin kx \rangle &= \frac{1}{\pi}\int_{-\pi}^\pi \sin^2 kx \, dx = 1 \end{align*} $$
(These integrals may be verified by using the product-to-sum trigonometric identities.)
Finally,
$$ \left\lVert \frac{1}{\sqrt{2}} \right\rVert^2 = \left\langle\ \frac{1}{\sqrt{2}},\ \frac{1}{\sqrt{2}} \right\rangle = \frac{1}{2\pi}\int_{-\pi}^\pi 1 \, dx = 1 $$
Therefore $\mathcal{F}_n$ is an orthonormal set. Define $\mathcal{F} = \bigcup_{n=1}^\infty \mathcal{F}_n$. Set $\mathcal{F}$ is called a Fourier Basis. A series can be defined using $\mathcal{F}$.
$$\frac{a_0}{2} + \sum_{n=1}^\infty \left[ a_n\cos(nx) + b_n\sin(nx) \right]$$
Note that it is customary to write the constant term $\dfrac{a_0}{2}$ instead of $\dfrac{a_0}{\sqrt{2}}$. We will explore this in a future section.
The topic of Fourier Series and their approximation by functions in $\mathrm{Span}(\mathcal{F}_n)$ is very important to physics and electrical engineering.
Given a basis for inner product space $\mathbb{E}^n$, let us compute the coefficients for vectors in $\mathbb{E}^n$ and their inner products.
Example 6 - Coordinates and Inner Products¶
Consider the basis $F = \left\{\,\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix},\ \begin{bmatrix} -2\ \\ \ 1\ \\ \ 3\ \end{bmatrix},\ \begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 2\ \end{bmatrix}\,\right\}$ in $\mathbb{E}^3$. The vectors in basis $F$ are neither orthogonal nor unit vectors. For any vector $\mathbf{x}\in\mathbb{E}^3$ given in standard coordinates such as $\mathbf{x} = \begin{bmatrix}\ 0\ \\ \ 4\ \\ \ 7\ \end{bmatrix}$, what are the $F$-coordinates of vector $\mathbf{x}$ in the $F$ coordinate system?
1. To compute these coordinates for $\mathbf{x}$ with respect to such a basis solve the linear system¶
$$ \begin{align*} F\mathbf{x}_F &= \mathbf{x} \\ \\ \begin{bmatrix}\ \ 1\ & -2\ &\ \ 0\ \\ \ \ 1\ &\ \ 1\ &\ \ 1\ \\ \ \ 1\ &\ \ 3\ &\ \ 2\ \end{bmatrix}\mathbf{x}_F &= \begin{bmatrix}\ 0\ \\ \ 4\ \\ \ 7\ \end{bmatrix} \\ \\ \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 0 &\ 0\ \\ \ \ 1\ &\ \ 1\ &\ \ 1 &\ 4\ \\ \ \ 1\ &\ \ 3\ &\ \ 2 &\ 7\ \end{array} \right] &\xrightarrow{\substack{R_2-R_1 \\ R_3-R_1}} \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 0 &\ 0\ \\ \ \ 0\ &\ \ 3\ &\ \ 1 &\ 4\ \\ \ \ 0\ &\ \ 5\ &\ \ 2 &\ 7\ \end{array} \right] \xrightarrow{R_3-R_2} \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 0 &\ 0\ \\ \ \ 0\ &\ \ 3\ &\ \ 1 &\ 4\ \\ \ \ 0\ &\ \ 2\ &\ \ 1 &\ 3\ \end{array} \right] \xrightarrow{R_2-R_3} \\ \\ \left[ \begin{array}{ccc|c}\ \ 1\ & -2\ &\ \ 0 &\ 0\ \\ \ \ 0\ &\ \ 1\ &\ \ 0 &\ 1\ \\ \ \ 0\ &\ \ 2\ &\ \ 1 &\ 3\ \end{array} \right] &\xrightarrow{\substack{R_1+2R_2 \\ R_3-2R_2}} \left[ \begin{array}{ccc|c}\ \ 1\ &\ \ 0\ &\ \ 0 &\ 2\ \\ \ \ 0\ &\ \ 1\ &\ \ 0 &\ 1\ \\ \ \ 0\ &\ \ 0\ &\ \ 1 &\ 1\ \end{array} \right] \end{align*} $$
Therefore we have that
$$ \mathbf{x}_F = 2\,\begin{bmatrix}\ 1\ \\ \ 1\ \\ \ 1\ \end{bmatrix} + 1\,\begin{bmatrix} -2\ \\ \ \ 1\ \\ \ \ 3\ \end{bmatrix} + 1\,\begin{bmatrix}\ 0\ \\ \ 1\ \\ \ 2\ \end{bmatrix} = \begin{bmatrix}\ 2\ \\ \ 1\ \\ \ 1\ \end{bmatrix}_F $$
2. What are the inner products of vector $\mathbf{x}$ with the basis vectors?¶
$$ \begin{align*} \mathbf{x}\cdot\langle 1, 1, 1 \rangle &= 0(1) + 4(1) + 7(1) = 11 \\ \\ \mathbf{x}\cdot\langle -2, 1, 3 \rangle &= 0(-2) + 4(1) + 7(3) = 25 \\ \\ \mathbf{x}\cdot\langle 0, 1, 2 \rangle &= 0(0) + 4(1) + 7(2) = 18 \\ \end{align*} $$
Compute the inner product using standard coordinates. This is very important when using this basis.
3. What are the scalar components of vector $\mathbf{x}$ with the basis vectors?¶
$$ \begin{align*} \mathrm{Comp}_{\mathbf{v}_1}\mathbf{x} &= \dfrac{\mathbf{x} \cdot \mathbf{v}_1}{\lVert \mathbf{v}_1 \rVert } = \dfrac{11}{\sqrt{3}} \\ \\ \mathrm{Comp}_{\mathbf{v}_2}\mathbf{x} &= \dfrac{\mathbf{x} \cdot \mathbf{v}_2}{\lVert \mathbf{v}_2 \rVert} = \dfrac{25}{\sqrt{14}} \\ \\ \mathrm{Comp}_{\mathbf{v}_3}\mathbf{x} &= \dfrac{\mathbf{x} \cdot \mathbf{v}_3}{\lVert \mathbf{v}_3 \rVert} = \dfrac{18}{\sqrt{5}} \\ \end{align*} $$
4. What are the vector projections of vector $\mathbf{x}$ onto each basis vector?¶
$$ \begin{align*} \mathrm{Proj}_{\mathbf{v_1}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{v}_1}{\langle\mathbf{v}_1,\mathbf{v}_1\rangle}\mathbf{v}_1 = \dfrac{11}{3}\mathbf{v}_1 \\ \\ \mathrm{Proj}_{\mathbf{v_2}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{v}_2}{\langle\mathbf{v}_2,\mathbf{v}_2\rangle}\mathbf{v}_2 = \dfrac{25}{14}\mathbf{v}_2 \\ \\ \mathrm{Proj}_{\mathbf{v_3}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{v}_3}{\langle\mathbf{v}_3,\mathbf{v}_3\rangle}\mathbf{v}_3 = \dfrac{18}{5}\mathbf{v}_3 \\ \end{align*} $$
Notice that with a basis that is neither orthogonal, nor composed of unit vectors, vectors $\mathbf{x}\in\mathbb{E}^3$ may have:
coordinates,
inner products with each basis vector $\mathbf{v}_j$, and
coefficients to projections onto each basis vector $\mathbf{v}_j$
that are all different !
Exercise 3 - Coordinates in a Non-orthogonal Basis¶
The set
$$ W = \left\{\, \begin{bmatrix}\ \ 1\ \\ \ \ 1\ \\ \ \ 0\ \end{bmatrix},\ \begin{bmatrix}\ \ 1\ \\ \ \ 0\ \\ \ \ 1\ \end{bmatrix},\ \begin{bmatrix}\ \ 0\ \\ \ \ 1\ \\ \ \ 1\ \end{bmatrix} \,\right\} $$
is a basis for $\mathbb{E}^3$ that is not orthogonal. Find the $W$-coordinates of $\mathbf{x} = (3,3,4)$.
Check Your Answer
Because $W$ is not orthogonal, solve the linear system $W\mathbf{x}_W = \mathbf{x}$, that is $c_1+c_2 = 3$, $c_1+c_3 = 3$, $c_2+c_3 = 4$. This gives $c_1 = 1$, $c_2 = 2$, $c_3 = 2$, so $$ \mathbf{x}_W = \begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ \ \ 2\ \end{bmatrix}. $$
Let us discover what happens when an orthonormal basis is chosen for $\mathbb{E}^3$.
Example 7 - Orthonormal Basis Coordinates¶
Consider the orthonormal basis derived from Example 2 and vector $\mathbf{x}\in\mathbb{E}^3$ given in standard coordinates by $\mathbf{x} = \begin{bmatrix}\ 0\ \\ \ 4\ \\ \ 7\ \end{bmatrix}$.
1. What are the coordinates of vector $\mathbf{x}$ in the $U = \left\{\mathbf{u}_1,\,\mathbf{u}_2,\,\mathbf{u}_3\right\}$ coordinate system?¶
$$ \begin{align*} U\mathbf{x}_U &= \mathbf{x} \\ \\ \begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ \\ -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ \frac{1}{\sqrt{6}}\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ &\ \ \frac{2}{\sqrt{6}}\ \end{bmatrix}\mathbf{x}_U &= \begin{bmatrix}\ 0\ \\ \ 4\ \\ \ 7\ \end{bmatrix} \\ \\ \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ &\ 0\ \\ -\frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ \frac{1}{\sqrt{6}}\ &\ 4\ \\ \ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ &\ \ \frac{2}{\sqrt{6}}\ &\ 7\ \end{array} \right] &\xrightarrow{\substack{R_2+R_1 \\ R_3-R_1}} \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}} &\ 0\ \\ \ \ 0\ &\ \ \frac{2}{\sqrt{2}}\ &\ \ 0 &\ 4\ \\ \ \ 0\ & -\frac{1}{\sqrt{2}}\ &\ \ \frac{3}{\sqrt{6}}\ &\ 7\ \end{array} \right] \xrightarrow{R_2/2} \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ &\ 0\ \\ \ \ 0\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 2\ \\ \ \ 0\ & -\frac{1}{\sqrt{2}}\ &\ \ \frac{3}{\sqrt{6}}\ &\ 7\ \end{array} \right] \xrightarrow{R_3+R_2} \\ \\ \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ &\ 0\ \\ \ \ 0\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 2\ \\ \ \ 0\ &\ \ 0\ &\ \ \frac{3}{\sqrt{6}}\ &\ 9\ \end{array} \right] &\xrightarrow{R_3\frac{\sqrt{6}}{3}} \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ &\ 0\ \\ \ \ 0\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ 3\sqrt{6}\ \end{array} \right] \xrightarrow{R_1+R_3/\sqrt{6}} \\ \\ \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 3\ \\ \ \ 0\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ 3\sqrt{6}\ \end{array} \right] &\xrightarrow{R_1-R_2} \left[ \begin{array}{ccc|c}\ \ \frac{1}{\sqrt{3}}\ &\ \ 0\ &\ \ 0\ &\ 1\ \\ \ \ 0\ &\ \ \frac{1}{\sqrt{2}}\ &\ \ 0\ &\ 2\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ 3\sqrt{6}\ \end{array} \right] \xrightarrow{\substack{R_1\sqrt{3} \\ R_2\sqrt{2}}} \left[ \begin{array}{ccc|c}\ \ 1\ &\ \ 0\ &\ \ 0\ &\ \sqrt{3}\ \\ \ \ 0\ &\ \ 1\ &\ \ 0\ &\ 2\sqrt{2}\ \\ \ \ 0\ &\ \ 0\ &\ \ 1\ &\ 3\sqrt{6}\ \end{array} \right] \end{align*} $$
Hence,
$$ \mathbf{x}_U = \sqrt{3}\,\begin{bmatrix}\ \ \frac{1}{\sqrt{3}}\ \\ -\frac{1}{\sqrt{3}}\ \\ \ \ \frac{1}{\sqrt{3}}\ \end{bmatrix} + 2\sqrt{2}\,\begin{bmatrix}\ \ \frac{1}{\sqrt{2}}\ \\ \ \ \frac{1}{\sqrt{2}}\ \\ \ \ 0\ \end{bmatrix} + 3\sqrt{6}\,\begin{bmatrix} -\frac{1}{\sqrt{6}}\ \\ \ \ \frac{1}{\sqrt{6}}\ \\ \ \ \frac{2}{\sqrt{6}}\ \end{bmatrix} = \begin{bmatrix}\ \ \sqrt{3}\ \\ \ \ 2\sqrt{2}\ \\ \ \ 3\sqrt{6}\ \end{bmatrix}_U $$
2. What are the inner products of vector $\mathbf{x}$ with the basis vectors?¶
$$ \begin{align*} \mathbf{x}\cdot\mathbf{u}_1 &= 0\left(\frac{1}{\sqrt{3}}\right) + 4\left(-\frac{1}{\sqrt{3}}\right) + 7\left(\frac{1}{\sqrt{3}}\right) = \sqrt{3} \\ \\ \mathbf{x}\cdot\mathbf{u}_2 &= 0\left(\frac{1}{\sqrt{2}}\right) + 4\left(\frac{1}{\sqrt{2}}\right) + 7\left(0\right) = 2\sqrt{2} \\ \\ \mathbf{x}\cdot\mathbf{u}_3 &= 0\left(-\frac{1}{\sqrt{6}}\right) + 4\left(\frac{1}{\sqrt{6}}\right) + 7\left(\frac{2}{\sqrt{6}}\right) = 3\sqrt{6} \\ \end{align*} $$
3. What are the scalar components of vector $\mathbf{x}$ with the basis vectors?¶
$$ \begin{align*} \mathrm{Comp}_{\mathbf{u}_1}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_1}{\lvert\mathbf{u}_1\rvert} = \dfrac{\sqrt{3}}{1} = \sqrt{3} \\ \\ \mathrm{Comp}_{\mathbf{u}_2}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_2}{\lvert\mathbf{u}_2\rvert} = \dfrac{2\sqrt{2}}{1} = 2\sqrt{2} \\ \\ \mathrm{Comp}_{\mathbf{u}_3}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_3}{\lvert\mathbf{u}_3\rvert} = \dfrac{3\sqrt{6}}{1} = 3\sqrt{6} \end{align*} $$
4. What are the vector projections of vector $\mathbf{x}$ onto each basis vector?¶
$$ \begin{align*} \mathrm{Proj}_{\mathbf{u_1}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_1}{\mathbf{u}_1\cdot\mathbf{u}_1}\mathbf{u}_1 = \dfrac{\sqrt{3}}{1}\mathbf{u}_1 = \sqrt{3}\,\mathbf{u}_1 \\ \\ \mathrm{Proj}_{\mathbf{u_2}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_2}{\mathbf{u}_2\cdot\mathbf{u}_2}\mathbf{u}_2 = \dfrac{2\sqrt{2}}{1}\mathbf{u}_2 = 2\sqrt{2}\,\mathbf{u}_2 \\ \\ \mathrm{Proj}_{\mathbf{u_3}}\mathbf{x} &= \dfrac{\mathbf{x}\cdot\mathbf{u}_3}{\mathbf{u}_3\cdot\mathbf{u}_3}\mathbf{u}_3 = \dfrac{3\sqrt{6}}{1}\mathbf{u}_3 = 3\sqrt{6}\,\mathbf{u}_3 \end{align*} $$
Theorem 2¶
Orthonormal Coordinates
If $Q = \left\{\, \mathbf{q}_1,\ \dots,\ \mathbf{q}_n \,\right\}$ is an orthonormal basis for finite dimensional inner product space $V$, then the $Q$-coordinates of $\mathbf{v}\in V$ are determined by
$$v_i = \langle \mathbf{v} ,\ \mathbf{q}_i \rangle$$
Thus
$$\mathbf{v}_Q = \begin{bmatrix}\ \langle \mathbf{v},\ \mathbf{q}_1 \rangle\ \\ \ \vdots\ \\ \langle \mathbf{v},\ \mathbf{q}_n \rangle\ \end{bmatrix} = \left[ \langle \mathbf{v},\ \mathbf{q}_i \rangle \right]$$
Proof of Theorem 2¶
Let $Q = \left\{\, \mathbf{q}_1,\ \dots,\ \mathbf{q}_n \,\right\}$ be an orthonormal basis for finite dimensional inner product space $V$, and $\mathbf{v}\in V$. Then $\mathbf{v}$ is a linear combination of the basis vectors in $Q$.
$$\mathbf{v} = \sum_{i=1}^n c_i\mathbf{q}_i$$
For $1\le j\le n$ compute the inner product
$$\langle \mathbf{v}, \mathbf{q}_j \rangle$$
Simplifying
$$\begin{align*} \langle \mathbf{v}, \mathbf{q}_j \rangle &= \left\langle \sum_{i=1}^n c_i\mathbf{q}_i, \mathbf{q}_j \right\rangle = \sum_{i=1}^n \left\langle c_i\mathbf{q}_i, \mathbf{q}_j \right\rangle = \sum_{i=1}^n c_i\left\langle \mathbf{q}_i, \mathbf{q}_j \right\rangle \\ \\ &= \sum_{i=1}^n c_i\delta_{ij} = c_j \end{align*}$$
Computing the coordinates of a vector $\mathbf{v}$ in the inner product space by solving a linear system is unnecessary. Each coordinate evaluates to the inner product $c_j = \langle \mathbf{v},\mathbf{q}_j \rangle$. $\blacksquare$
Therefore we will almost always use an orthonormal basis in our mathematical models.¶
Exercise 4 - Coordinates in an Orthonormal Basis¶
Let $Q$ be the orthonormal basis for $\mathbb{E}^3$
$$ Q = \left\{\, \frac{1}{3}\begin{bmatrix}\ \ 2\ \\ \ \ 1\ \\ \ \ 2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}-2\ \\ \ \ 2\ \\ \ \ 1\ \end{bmatrix} \,\right\}. $$
Use inner products to find the $Q$-coordinates of $\mathbf{x} = (3,0,3)$.
Check Your Answer
Since $Q$ is orthonormal, each coordinate is an inner product: $\langle \mathbf{x},\mathbf{q}_1 \rangle = \frac{1}{3}(6+0+6) = 4$, $\langle \mathbf{x},\mathbf{q}_2 \rangle = \frac{1}{3}(3+0-6) = -1$, and $\langle \mathbf{x},\mathbf{q}_3 \rangle = \frac{1}{3}(-6+0+3) = -1$. Hence $$ \mathbf{x}_Q = \begin{bmatrix}\ \ 4\ \\ -1\ \\ -1\ \end{bmatrix}. $$
Consider Euclidean inner product space $\mathbb{R}^n$. The definition of dot product relies on the familiar geometry of projections. The value of the inner product of two vectors $\mathbf{u},\ \mathbf{v}\in\mathbb{R}^n$ can be computed
$$ \mathbf{u}\cdot\mathbf{v} = \lvert\mathbf{u}\rvert\, \lvert\mathbf{v}\rvert\,\cos(\theta) $$
This formula makes no mention of the coordinates of the vectors $\mathbf{u}$ and $\mathbf{v}$.
Can one compute the correct value of the inner product of two vectors using their non-standard representation?
Usually one must convert the coordinates of the vector to standard coordinates first. To compute the inner product $\langle\cdot,\cdot\rangle_F$ in $F$-coordinates
$$ \left\langle\left[\mathbf{x}\right]_F,\,\left[\mathbf{y}\right]_F\right\rangle_F = \left\langle F\left[\mathbf{x}\right]_F,\,F\left[\mathbf{y}\right]_F\right\rangle = \langle\mathbf{x},\,\mathbf{y}\rangle $$
However if one chooses an orthonormal basis, computation is much simpler.
Theorem 3¶
Invariance of Inner Product
Let $Q = \left\{\, \mathbf{q}_1,\ \ldots,\ \mathbf{q}_n\right\}$ be an orthonormal basis for a finite dimensional inner product space $V$. If
$$ \mathbf{x} = \sum_{i=1}^n x_i\mathbf{q}_i \qquad\qquad \mathbf{y} = \sum_{i=1}^n y_i\mathbf{q}_i $$
then
$$ \langle \mathbf{x},\mathbf{y} \rangle = \sum_{i=1}^n x_i y_i. $$
Proof of Theorem 3¶
Let $Q = \left\{\mathbf{q}_1,\mathbf{q}_2,\ldots,\mathbf{q}_n\right\}$ be an orthonormal basis for a finite dimensional inner product space $V$, and let $\mathbf{x}$, $\mathbf{y}\in V$. From Theorem 2 we have that coordinates of vector can be computed using the inner product.
$$ \begin{align*} \langle \mathbf{x},\mathbf{q}_i \rangle = x_i \qquad i=1,\dots,n \\ \langle \mathbf{y},\mathbf{q}_i \rangle = y_i \qquad i=1,\dots,n \\ \end{align*} $$
Using the properties of inner product on obtains:
$$ \begin{align*} \langle \mathbf{x},\mathbf{y} \rangle &= \left\langle \sum_{i=1}^n x_i\mathbf{q}_i, \mathbf{y} \right\rangle = \sum_{i=1}^n x_i \langle \mathbf{q}_i, \mathbf{y} \rangle = \sum_{i=1}^n x_i \left\langle\ \mathbf{q}_i, \sum_{j=1}^n y_j \mathbf{q}_j \right\rangle \\ \\ &= \sum_{i=1}^n x_i \sum_{j=1}^n y_j \langle\ \mathbf{q}_i, \mathbf{q}_j \rangle = \sum_{i=1}^n \sum_{j=1}^n x_iy_j \delta_{ij} = \sum_{i=1}^n x_i y_i \qquad\blacksquare \end{align*} $$
We have already seen that if we choose a different orientation (basis or coordinate system), then the coordinates (list of numbers) that describe each vector are also different.
However the value of the inner product of two vectors in an inner product space does not change. When one defines an inner product on a vector space, the value of the inner product does NOT depend on the choice of orientation, or choice of basis. Therefore we will always choose an orthonormal basis with respect to the inner product so that this invariant property (nonchanging property) can be easily computed.
Example 8 - Compute Inner Product with Different Bases¶
Consider the orthonormal basis derived from Example 2 and vectors $\mathbf{x},\ \mathbf{y}\in\mathbb{E}^3$ given in standard coordinates by
$$\mathbf{x} = \begin{bmatrix}\ \ 0\ \\ \ \ 4\ \\ \ \ 7\ \end{bmatrix} \qquad\text{and}\qquad \mathbf{y} = \begin{bmatrix}\ \ 2\ \\ \ \ 3\ \\ -3\ \end{bmatrix}. $$
$$ \mathbf{u}_1 = \begin{bmatrix}\ \ \frac{1}{\sqrt{3}} \\ -\frac{1}{\sqrt{3}} \\ \ \ \frac{1}{\sqrt{3}} \end{bmatrix} \qquad\qquad \mathbf{u}_2 = \begin{bmatrix} \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \\ 0 \end{bmatrix} \qquad\qquad \mathbf{u}_3 = \begin{bmatrix} -\frac{1}{\sqrt{6}} \\ \ \ \frac{1}{\sqrt{6}} \\ \ \ \frac{2}{\sqrt{6}}\ \end{bmatrix} $$
Using the results from Theorem 2 and Example 7,
$$ \mathbf{x}_U = \begin{bmatrix}\ \ \sqrt{3}\ \\ \ \ 2\sqrt{2}\ \\ \ \ 3\sqrt{6}\ \end{bmatrix}_U \qquad\qquad \mathbf{y}_U = \begin{bmatrix} \langle \mathbf{y}, \mathbf{u}_1 \rangle \\ \langle \mathbf{y}, \mathbf{u}_2 \rangle \\ \langle \mathbf{y}, \mathbf{u}_3 \rangle \end{bmatrix}_U = \begin{bmatrix} -\frac{4}{\sqrt{3}}\ \\ \ \ \frac{5}{\sqrt{2}}\ \\ -\frac{5}{\sqrt{6}}\ \end{bmatrix}_U $$
Now compute inner product with respect to each basis.
$$ \begin{align*} \mathbf{x}\cdot\mathbf{y} &= 0(2) + 4(3) + 7(-3) = 0 + 12 - 21 = -9 \\ \\ \mathbf{x}_U\cdot\mathbf{y}_U &= \sqrt{3}\left( -\frac{4}{\sqrt{3}} \right) + 2\sqrt{2}\left( \frac{5}{\sqrt{2}} \right) + 3\sqrt{6}\left( -\frac{5}{\sqrt{6}} \right) = -4 + 10 - 15 = -9\ {\color{green}\Large{\checkmark}} \end{align*}$$
Exercise 5 - Invariance of the Inner Product¶
Let $Q$ be the orthonormal basis for $\mathbb{E}^3$
$$ Q = \left\{\, \frac{1}{3}\begin{bmatrix}\ \ 2\ \\ \ \ 1\ \\ \ \ 2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}\ \ 1\ \\ \ \ 2\ \\ -2\ \end{bmatrix},\ \frac{1}{3}\begin{bmatrix}-2\ \\ \ \ 2\ \\ \ \ 1\ \end{bmatrix} \,\right\}. $$
For $\mathbf{x} = (3,0,3)$ and $\mathbf{y} = (0,3,3)$, compute $\langle \mathbf{x},\mathbf{y} \rangle$ in standard coordinates and verify that it equals $\langle \mathbf{x}_Q,\mathbf{y}_Q \rangle$.
Check Your Answer
In standard coordinates $\langle \mathbf{x},\mathbf{y} \rangle = 0 + 0 + 9 = 9$. The $Q$-coordinates are $\mathbf{x}_Q = (4,-1,-1)$ and $\mathbf{y}_Q = (3,0,3)$, so $\langle \mathbf{x}_Q,\mathbf{y}_Q \rangle = 12 + 0 - 3 = 9$. The two agree, as guaranteed for an orthonormal basis.
Theorem 4¶
Parseval's Identity
If $Q = \left\{\, \mathbf{q}_1,\ \ldots,\ \mathbf{q}_n \,\right\}$ is an orthonormal basis for a finite dimensional inner product space $V$ and $\mathbf{x}\in V$ such that
$$ \mathbf{x} = \sum_{i=1}^n x_i\mathbf{q}_i $$
then
$$ \lVert \mathbf{x} \rVert_Q^2 = \langle \mathbf{x},\mathbf{x} \rangle_Q = \sum_{i=1}^n x_i^2 $$
Parseval's Identity holds for infinite dimensional inner product spaces as well. However proving this identity for infinite dimensional vector spaces is behyond the scope of this course.
Proof of Theorem 4¶
Suppose that $Q = \left\{\, \mathbf{q}_1,\ \ldots,\ \mathbf{q}_n \,\right\}$ is an orthonormal basis for a finite dimensional inner product space $V$ and $\mathbf{x}\in V$ such that
$$ \mathbf{x} = \sum_{i=1}^n x_i\mathbf{q}_i. $$
Then using Theorem 3,
$$ \lVert \mathbf{x} \rVert_Q^2 = \langle \mathbf{x},\mathbf{x} \rangle_Q = \sum_{i=1}^n x_i^2 \qquad \blacksquare $$
Exercise 6 - Compute Norms¶
Consider again orthonormal basis $\mathcal{F}_n = \left\{\frac{1}{\sqrt{2}},\cos x,\sin x,\cos 2x,\,\sin 2x,\ldots,\cos nx,\sin nx\right\}$ in $C[-\pi,\pi]$ and the finite dimensional subspace
$$ V = \mathrm{Span}(\mathcal{F}_n) $$
Compute the following using the Fourier basis $\mathcal{F}_n$ and orthogonality.
- $ \lVert \sin^2 x \rVert^2 $
- $ \lVert 2\sin 3x + \sin^2 x \rVert^2$
- $ \lVert 2 + \cos^4 x\rVert^2 $
Check Your Work
$$ \begin{align*} \lVert \sin^2 x \rVert^2 &= \frac{3}{4} \\ \\ \lVert 2\sin 3x + \sin^2 x \rVert^2 &= \frac{19}{4} \\ \\ \lVert 2 + \cos^4 x \rVert^2 &= \frac{739}{64} \end{align*} $$
Follow Along
We could integrate these directly, but that would not take advantage of our new powerful tool Parseval's identity. Instead, we shall employ trigonometric identities to write each of these functions in terms of our orthonormal basis and then apply Parseval's identity to find the (square of the) norm.
$\lVert \sin^2 x \rVert^2 = \left\lVert \frac{1}{2}\left(1-\cos 2x\right) \right\rVert^2 = \left\lVert {\color{crimson}\frac{1}{\sqrt{2}}}\cdot{\color{royalblue}\frac{1}{\sqrt{2}}} - {\color{crimson}\frac{1}{2}}{\color{royalblue}\cos 2x} \right\rVert^2 = \left( {\color{crimson}\frac{1}{\sqrt{2}}} \right)^2 + \left( {\color{crimson}-\frac{1}{2}} \right)^2 = \frac{3}{4}$
$\lVert 2 \sin 3x + \sin^2 x \rVert^2 = \left\lVert {\color{crimson}\frac{1}{\sqrt{2}}}\cdot{\color{royalblue}\frac{1}{\sqrt{2}}} {\color{crimson}- \frac{1}{2}}{\color{royalblue}\cos 2x} + {\color{crimson}2}{\color{royalblue}\sin 3x} \right\rVert^2 = \left( {\color{crimson}\frac{1}{\sqrt{2}}} \right)^2 + \left( {\color{crimson}-\frac{1}{2}} \right)^2 + \left( {\color{crimson}2} \right)^2 = \frac{19}{4}$
$\lVert 2 + \cos^4 x\rVert^2 = \left\lVert 2 + \left[ \frac{1}{2}\left(1+\cos 2x\right)\right]^2 \right\rVert^2 = \left\lVert 2 + \frac{1}{4}\left(1+2\cos 2x + \cos^2 2x \right) \right\rVert^2$
$= \left\lVert \frac{9}{4} + \frac{1}{2}\cos 2x + \frac{1}{8}\left(1+\cos 4x \right) \right\rVert^2 = \left\lVert \frac{19}{8} + \frac{1}{2}\cos 2x + \frac{1}{8}\cos 4x \right\rVert^2$
$= \left\lVert {\color{crimson}\frac{19\sqrt{2}}{8}} \cdot {\color{royalblue}\frac{1}{\sqrt{2}}} + {\color{crimson}\frac{1}{2}}{\color{royalblue}\cos 2x} + {\color{crimson}\frac{1}{8}}{\color{royalblue}\cos 4x} \right\rVert^2 = \left( {\color{crimson}\frac{19\sqrt{2}}{8}} \right)^2 + \left( {\color{crimson}\frac{1}{2}} \right)^2 + \left( {\color{crimson}\frac{1}{8}} \right)^2 = \frac{739}{64} $
Example 9 - Compute Integrals¶
Compute the following integrals using Parseval's identity and orthogonality.
$$ \begin{align*} \displaystyle\int_{-\pi}^{\pi}\sin^4(x)\,dx &= \dfrac{\pi}{\pi}\displaystyle\int_{-\pi}^{\pi}\sin^2(x)\sin^2(x)\,dx \\ \\ &= \pi\,\left\lVert\sin^2(x)\right\rVert^2 = \dfrac{3\pi}{4} \\ \\ \displaystyle\int_{-\pi}^{\pi}\left(2\sin 3x + \sin^2 x\right)^2\, dx &= \dfrac{\pi}{\pi}\displaystyle\int_{-\pi}^{\pi}\left(2\sin 3x + \sin^2 x\right)^2\, dx \\ \\ &= \pi\,\left\lVert 2\sin 3x + \sin^2 x \right\rVert^2 = \frac{19\pi}{4} \\ \\ \displaystyle\int_{-\pi}^{\pi}\left(2 + \cos^4 x\right)^2\, dx &= \dfrac{\pi}{\pi}\displaystyle\int_{-\pi}^{\pi}\left(2 + \cos^4 x\right)^2\, dx \\ \\ &= \pi\,\left\lVert 2 + \cos^4 x \right\rVert^2 = \frac{739\pi}{64} \end{align*} $$
Exercise 7¶
Compute the following integral using linear algebra.
$$ \displaystyle\int_0^{\pi} \cos^3(t)\,dt $$
Check Your Work
$$ \displaystyle\int_0^{\pi} \cos^3(t)\,dt = 0 $$
Follow Along
The function $\cos^3(t)$ is an *even* function, that is $\cos^3(-t) = \cos^3(t)$, so its graph is symmetric with respect to the $y$-axis. Thus the areas on the left and right of the $y$-axis are mirror images. Thus $$ \begin{align*} \displaystyle\int_0^{\pi} \cos^3(t)\,dt &= \dfrac{1}{2}\displaystyle\int_{-\pi}^{\pi} \cos^3(t)\,dt = \dfrac{1}{2}\displaystyle\int_{-\pi}^{\pi} \cos(t)\cos^2(t)\,dt \\ \\ &= \dfrac{1}{2}\displaystyle\int_{-\pi}^{\pi} \cos(t)\dfrac{1 + \cos(2t)}{2}\,dt \\ \\ &= \dfrac{1}{4}\displaystyle\int_{-\pi}^{\pi} \left(\cos(t) + \cos(t)\cos(2t)\right)\,dt \\ \\ &= \dfrac{1}{4}\displaystyle\int_{-\pi}^{\pi} \cos(t)\,dt + \dfrac{1}{4}\displaystyle\int_{-\pi}^{\pi} \cos(t)\cos(2t)\,dt \\ \\ &= \dfrac{\pi}{4}\,\dfrac{1}{\pi}\displaystyle\int_{-\pi}^{\pi} 1\cdot\cos(t)\,dt + \dfrac{\pi}{4}\,\dfrac{1}{\pi}\displaystyle\int_{-\pi}^{\pi} \cos(t)\cos(2t)\,dt \\ \\ &= \dfrac{\pi}{4}\,\left\langle 1,\,\cos(t)\,\right\rangle + \dfrac{\pi}{4}\,\left\langle\, \cos(t),\,\cos(2t)\,\right\rangle = 0. \end{align*} $$
Exercise 8 - Orthonormal Columns of a Square Matrix¶
If the columns of a square matrix $A\in\mathbb{R}^{n\times n}$ form an orthonormal set, then the columns must form an orthonormal basis for $\mathbb{R}^n$. Why?
Check Your Answer
If the columns of a square matrix $A\in\mathbb{R}^{n\times n}$ form an orthonormal set, then they are a set of $n$ linearly independent vectors in $\mathbb{R}^n$. The columns of matrix $A$ must form a basis. As these vectors are orthonormal, they form an orthonormal basis for $\mathbb{R}^n$, the domain and the codomain of the linear transformation represented by $A$.
Definition¶
Orthogonal Matrix
A matrix $Q\in\mathbb{R}^{n\times n}$ is an orthogonal matrix if and only if its columns are an orthonormal set in $\mathbb{R}^n$.
Clearly, this means that such a matrix would be the transition matrix from the column basis, a basis for the column space $C(A)$ of matrix $A$
$$ \left\{\, \mathbf{q}_1,\ \ldots,\ \mathbf{q}_n \,\right\} $$
to the standard (canonical) basis
$$ \left\{\, \mathbf{e}_1,\ ,\ldots,\ \mathbf{e}_n \,\right\}. $$
The transition matrix $Q^{-1}$ from $\left\{\mathbf{e}_1,\mathbf{e}_2,\ldots,\mathbf{e}_n\right\}$ to $\left\{\mathbf{q}_1,\mathbf{q}_2,\ldots,\mathbf{q}_n\right\}$ has a special form for orthogonal matrices.
Theorem 5¶
Properties of Orthogonal Matrices
If $Q\in\mathbb{R}^{n\times n}$, then the following are equivalent:
$Q$ is an orthogonal matrix
The columns of $Q$ are an orthonormal basis for $\mathbb{R}^n$
$ Q^T Q = I $
$ Q^T = Q^{-1} $
$ \left\langle Q\mathbf{x},Q\mathbf{y} \right\rangle = \langle \mathbf{x},\mathbf{y}\rangle$ for all $\mathbf{x}$ and $\mathbf{y}\in\mathbb{R}^n$
$\lVert Q\mathbf{x} \rVert = \lVert \mathbf{x} \rVert$ for all $\mathbf{x}\in\mathbb{R}^n$.
Moreover if $Q\in\mathbb{R}^{n\times n}$ is an orthogonal matrix, then
- $\det(Q) = \pm 1$
Proof of Theorem 5¶
$\implies$2. If matrix $Q\in\mathbb{R}^n$ is an orthogonal matrix, then the columns of matrix $Q$ form an orthonormal set of $n$ vectors. Therefore the columns of $Q$ form an orthonormal basis for $\mathbb{R}^n$.
$\implies$3. If the columns of $n\times n$ matrix $Q$ form an orthonormal basis for $\mathbb{R}^n$, then each element of $C=Q^TQ$ is the product of the $i^\text{th}$ row of $Q^T$ and $j^\text{th}$ column of $Q$. $$ c_{ij} = \mathbf{q}_i^T\mathbf{q}_j = \langle \mathbf{q}_i,\mathbf{q}_j \rangle = \delta_{ij} $$ Hence, $$ C = \begin{bmatrix} c_{ij} \end{bmatrix} = \begin{bmatrix} \delta_{ij} \end{bmatrix} = I. $$ That is $Q^T Q = I$.
$\implies$4. If $Q\in\mathbb{R}^{n\times n}$ and $Q^TQ=I$, then $Q^T = Q^{-1}$ since inverses are unique.
$\implies$5. If matrix $Q\in\mathbb{R}^{n\times n}$ has the property that $Q^T = Q^{-1}$, and $\mathbf{x}$ and $\mathbf{y}\in\mathbb{R}^n$, then $$\langle Q\mathbf{x},Q\mathbf{y} \rangle = \left( Q\mathbf{y} \right)^T\left( Q\mathbf{x} \right) = \mathbf{y}^TQ^TQ\mathbf{x} = \mathbf{y}^TQ^{-1}Q\mathbf{x} = \mathbf{y}^T\mathbf{x} = \langle \mathbf{x},\mathbf{y} \rangle. $$
$\implies$6. Suppose $Q\in\mathbb{R}^{n\times n}$ and for all vectors $\mathbf{x}$ and $\mathbf{y}\in\mathbb{R}^n$, $\langle Q\mathbf{x},Q\mathbf{y} \rangle = \langle \mathbf{x},\mathbf{y} \rangle$. Then $$\lVert Q\mathbf{x} \rVert^2 = \langle Q\mathbf{x},Q\mathbf{x} \rangle = \langle\mathbf{x},\mathbf{x}\rangle = \lvert\mathbf{x}\rvert^2.$$ Computing the square root of both sides of this equation yields $\lvert Q\mathbf{x} \rvert = \lvert \mathbf{x} \rvert$ for all $\mathbf{x}\in\mathbb{R}^n$.
$\implies$1. Suppose $Q\in\mathbb{R}^n$ and for all vectors $\mathbf{x}, \mathbf{y}\in\mathbb{R}^n$, $\lvert Q\mathbf{x},Q\mathbf{y} \rvert = \lvert \mathbf{x} \rvert$. Consider the canonical basis for $\mathbb{R}^n$, $\left\{\mathbf{e}_1, \dots, \mathbf{e}_n \right\}$. For every $1\le i\le n$, $Q\mathbf{e}_i = \mathbf{q}_i$, the $i^{\text{th}}$ column of matrix $Q$. Hence, for every $1 \le i,j \le n$, $$\langle \mathbf{q}_i,\mathbf{q}_j \rangle = \langle Q\mathbf{e}_i, Q\mathbf{e}_j \rangle = \langle\mathbf{e}_i,\mathbf{e}_j \rangle = \delta_{ij}.$$ Thus the columns of matrix $Q$ form an orthonormal set of $n$ vectors in $\mathbb{R}^n$. Therefore, $Q$ is an orthogonal matrix.
If $Q\in\mathbb{R}^{n\times n}$ is an orthogonal matrix, then $$ 1 = \det\left(I_n\right) = \det\left(Q^TQ\right) = \det(Q^T)\det(Q) = \det(Q)^2 \qquad\qquad\blacksquare$$
We say that orthogonal matrices preserve the inner product since $$ \left\langle Q\mathbf{x},Q\mathbf{y} \right\rangle = \langle \mathbf{x},\mathbf{y}\rangle $$
We say that orthogonal matrices preserve the norm since the linear transformation represented by matrix $Q$ maps vectors in $\mathbb{R}^n$ to vectors in $\mathbb{R}^n$ with the same norm. $$ \left\lVert Q\mathbf{x} \right\rVert = \left\lVert \mathbf{x} \right\rVert $$
Since the angle $\theta$ between two vectors $\mathbf{x},\mathbf{y}\in\mathbb{R}^n$ is the same as the angle between $Q\mathbf{x}$ and $Q\mathbf{y}$, we say that orthogonal matrices preserve angles.
$$ \cos(\theta) = \dfrac{\langle \mathbf{x},\mathbf{y} \rangle}{\lVert \mathbf{x} \rVert\, \lVert \mathbf{y} \rVert} = \dfrac{ \langle Q\mathbf{x},Q\mathbf{y} \rangle}{ \lVert Q\mathbf{x} \rVert\, \lVert Q\mathbf{y} \rVert}.$$
Several linear operators on $\mathbb{R}^n$ with clear geometric interpretations are expressed as orthogonal matrices. These include
- reflections
- rotations
- permutations
Definition¶
Reflection
A reflection is a linear operator on $\mathbb{E}^n$ that fixes an $n-1$ dimensional (hyper)plane through the origin and "flips" or "mirrors" all vectors on either side of that (hyper)plane. In $\mathbb{R}^2$ the hyperplane of reflection is a line and in $\mathbb{R}^3$ it is a two-dimensional plane. A plane in $\mathbb{R}^3$ may be defined by its normal vector, a vector that is orthogonal to each vector in the plane. Using this normal vector, we can construct the matrix representing the linear operator for a reflection across this plane.
Definition¶
Householder Reflection
The matrix representing the linear operator $R\,:\,\mathbb{R}^n\rightarrow\mathbb{R}^n$ defined by
$$ R := I - 2\mathbf{n}\mathbf{n}^T $$
for a unit vector $\mathbf{n}\in\mathbb{R}^n$ is called a Householder transformation or Householder Reflection.
Example 10 - Householder Reflection¶
Suppose we have a unit normal vector to a plane.
$$ \mathbf{n} = \left[\begin{array}{r} -\frac{1}{\sqrt{3}} \\ -\frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{array}\right] $$
Then we can define the plane $H$ in $\mathbb{R}^3$ as the graph of the equation
$$ \mathbf{n}\cdot\mathbf{x} = 0. $$
The length $\lvert\mathbf{n}\rvert = 1$ because we chose vector $\mathbf{n}$ to be a unit vector. Thus $\left\{ \mathbf{n} \right\}$ is an orthonormal set, and $P = \mathbf{n}\mathbf{n}^T$ is the projection of $\mathbb{R}^{n}$ onto the subspace $S = \mathrm{Span}\left\{\mathbf{n}\right\}$. Furthermore $I_n - P = I_n - \mathbf{n}\mathbf{n}^T$ is the projection of $\mathbb{R}^n$ onto the orthogonal complement of $S$. $H = S^{\perp}$, the hyperplane, and $S$ is the normal line to the hyperplane $H$.
We can also define a Householder reflection $R\,:\,\mathbb{R}^n\rightarrow\mathbb{R}^n$ by
$$ R = I - 2\mathbf{n}\mathbf{n}^T = \left[\begin{array}{rrr} \frac{1}{3} & -\frac{2}{3} & \frac{2}{3} \\ -\frac{2}{3} & \frac{1}{3} & \frac{2}{3} \\ \frac{2}{3} & \frac{2}{3} & \frac{1}{3} \end{array} \right] $$
The image below shows the effect of this reflection on a polygon $p$ defined by the points $(1,1.5,0.5),(0.5,1.5,1.5),(-1,2,0.5)$ plotted in $\mathbb{R}^3$, and reflected about the plane $H$, with the reflection denoted as $R_p$.
Theorem 6¶
Householder Reflections are Orthogonal Transformations
A Householder transformation is an orthogonal transformation
Proof of Theorem 6¶
$$ \begin{align*} RR^T &= \left(I - 2\mathbf{n}\mathbf{n}^T\right)\left(I - 2\mathbf{n}\mathbf{n}^T\right)^T = \left(I - 2\mathbf{n}\mathbf{n}^T\right)\left(I^T - \left(2\mathbf{n}\mathbf{n}^T\right)^T\right) \\ \\ &= \left(I - 2\mathbf{n}\mathbf{n}^T\right)\left(I - 2\mathbf{n}\mathbf{n}^T\right) = I - 2\mathbf{n}\mathbf{n}^T - 2\mathbf{n}\mathbf{n}^T + 4\left(\mathbf{n}\mathbf{n}^T\right)\left(\mathbf{n}\mathbf{n}^T\right) \\ \\ &= I - 4\mathbf{n}\mathbf{n}^T + 4\mathbf{n}\left(\mathbf{n}^T\mathbf{n}\right)\mathbf{n}^T = I - 4\mathbf{n}\mathbf{n}^T + 4\mathbf{n}(1)\mathbf{n}^T = I \end{align*} $$
As the $RR^T=I$, the matrix $R$ is invertible and $R^{-1}=R^T$, so matrix $R$ is an orthogonal matrix. $\blacksquare$
Exercise 9 - Constructing a Householder Reflection¶
Let $\mathbf{n} = \frac{1}{3}(2,-1,2)$, a unit normal vector to a plane $H$ in $\mathbb{R}^3$. Construct the Householder reflection $R = I - 2\mathbf{n}\mathbf{n}^T$ and verify that $R\mathbf{n} = -\mathbf{n}$.
Check Your Answer
$$ R = I - 2\mathbf{n}\mathbf{n}^T = \frac{1}{9}\begin{bmatrix}\ \ 1\ & \ \ 4\ & -8\ \\ \ \ 4\ & \ \ 7\ & \ \ 4\ \\ -8\ & \ \ 4\ & \ \ 1\ \end{bmatrix}. $$ Since $\mathbf{n}$ is a unit vector, $\mathbf{n}^T\mathbf{n} = 1$, so $R\mathbf{n} = \mathbf{n} - 2\mathbf{n}(\mathbf{n}^T\mathbf{n}) = \mathbf{n} - 2\mathbf{n} = -\mathbf{n}$; the normal direction is flipped while every vector in $H$ (orthogonal to $\mathbf{n}$) is fixed.
Definition¶
Rotation
A rotation is a linear operator that fixes a line in space and transforms all other vectors invariantly#Invariant_subsets) about that line by rotating all vectors around the line. This means
that although vectors are moved from one place in the coordinate system to another,
a rotation preserves the norm
a rotation preserves angles
This tells us that a rotation is an orthogonal linear transformation and its matrix representation is an orthogonal matrix.
Example 11 - Counterclockwise Rotation¶
Create the standard matrix representation of a rotation in $\mathbb{R}^2$ counterclockwise $\frac{\pi}{3}$ radians.
In order to create the standard matrix representation $R_{\pi/3}$ one needs only know where $\ihat$ and $\jhat$ are mapped by the rotation. On the unit circle
$$ \ihat\longrightarrow\begin{bmatrix}\ \cos\left(\frac{\pi}{3}\right)\ \\ \ \sin\left(\frac{\pi}{3}\right)\ \end{bmatrix}\qquad\qquad\jhat\longrightarrow\begin{bmatrix} -\sin\left(\frac{\pi}{3}\right)\ \\ \ \cos\left(\frac{\pi}{3}\right)\ \end{bmatrix} $$
Hence the standard matrix representation of the rotation is given by
$$ R_{\pi/3} = \begin{bmatrix}\ \frac{1}{2}\ & -\frac{\sqrt{3}}{2}\ \\ \ \frac{\sqrt{3}}{2}\ &\ \frac{1}{2}\ \end{bmatrix} $$
Example 12 - Clockwise Rotation¶
Create the standard matrix representation of linear transformation that rotates all vectors in $\mathbb{R}^3$ clockwise $\frac{\pi}{4}$ radians about the $x$-axis.
To determine the standard matrix representation of this rotation $R_{-\pi/4}$ we need the images of our canonical vectors
$$ \ihat \longrightarrow \ihat\qquad\jhat \longrightarrow \begin{bmatrix}\ 0\ \\ \ \cos\left(\frac{\pi}{4}\right)\ \\ -\sin\left(\frac{\pi}{4}\right)\ \end{bmatrix}\qquad\khat \longrightarrow \begin{bmatrix}\ 0\ \\ \ \sin\left(\frac{\pi}{4}\right)\ \\ \ \cos\left(\frac{\pi}{4}\right)\ \end{bmatrix} $$
Thus the standard matrix representation of our rotation about the $x$-axis is given by
$$ R_{-\pi/4} = \begin{bmatrix} \ 1\ & \ 0\ & \ 0\ \\ \ 0\ &\ \frac{1}{\sqrt{2}}\ &\ \frac{1}{\sqrt{2}}\ \\ \ 0\ & -\frac{1}{\sqrt{2}}\ &\ \frac{1}{\sqrt{2}}\ \end{bmatrix} $$
Example 13 - Composite Rotation¶
If we form a matrix $U$ using $\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3$ from Example 2, $U$ represents a rotation in $\mathbb{R}^3$. In fact, it may be written as a product of two rotations: one about the $y$-axis and one about the $z$-axis. Since $U$ is an orthogonal matrix, this decomposition may be determined using Euler angles. Employing the technique from the linked site, the rotation angles about the $x_2$- and $x_3$-axes are determined to be approximately $\theta=-0.6155$ and $\psi=-0.7854$ radians respectively. Since $U$ is the transition matrix from $\left\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\right\}$ to the standard basis the decomposition has the form
$$ \begin{align*} U &= \left[\begin{array}{rrr}\ \frac{1}{\sqrt{3}}\ &\ \frac{1}{\sqrt{2}}\ & -\frac{1}{\sqrt{6}}\ \\ -\frac{1}{\sqrt{3}}\ &\ \frac{1}{\sqrt{2}}\ &\ \frac{1}{\sqrt{6}}\ \\ \ \frac{1}{\sqrt{3}}\ &\ 0\ &\ \frac{2}{\sqrt{6}}\ \end{array}\right] = \\ \\ &= \left(R_y R_z\right)^{-1} \\ \\ &= \left( \left[ \begin{array}{ccc}\ \cos\theta\ &\ 0\ & -\sin\theta\ \\ \ 0\ &\ 1\ &\ 0\ \\ \ \sin\theta\ &\ 0\ &\ \cos\theta\ \end{array} \right] \left[ \begin{array}{ccc}\ \cos\psi\ &\ \sin\psi\ &\ 0\ \\ -\sin\psi\ & \ \cos\psi\ &\ 0\ \\ \ 0\ &\ 0\ &\ 1\ \end{array} \right] \right)^{-1} \\ \\ &= R_z^T R_y^T \end{align*} $$
since $R_z$ and $R_y$ are themselves both orthogonal matrices.
Both the original and rotated axes are shown here:
Exercise 10 - Constructing a Rotation Matrix¶
Find the standard matrix representation of the linear operator that rotates all vectors in $\mathbb{R}^3$ counterclockwise $\frac{\pi}{2}$ radians about the $z$-axis, and verify that it is an orthogonal matrix with determinant $1$.
Check Your Answer
The images of the canonical vectors are $\ihat \to (0,1,0)$, $\jhat \to (-1,0,0)$, $\khat \to (0,0,1)$, giving $$ R = \begin{bmatrix}\ \ 0\ & -1\ & \ \ 0\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & \ \ 0\ & \ \ 1\ \end{bmatrix}. $$ Then $R^T R = I$, so $R$ is orthogonal, and $\det(R) = 1$, confirming it is a rotation.
These exercises combine the ideas of this section with material from earlier chapters. Recall that a permutation matrix (Section 2.4) is obtained by permuting the columns of the identity matrix; its columns are the standard basis vectors in some order.
Exercise 11 - Permutation Matrices are Orthogonal¶
Let $P$ be the permutation matrix for the cyclic permutation $\mathbf{e}_1\to\mathbf{e}_2\to\mathbf{e}_3\to\mathbf{e}_1$.
- Write down $P$.
- Show that $P$ is an orthogonal matrix.
- Compute $\det(P)$ and state whether $P$ preserves or reverses orientation.
Check Your Answer
The columns of $P$ are the images of the standard basis vectors, so $$ P = \begin{bmatrix}\ \ 0\ & \ \ 0\ & \ \ 1\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & \ \ 1\ & \ \ 0\ \end{bmatrix}. $$ The columns are $\mathbf{e}_2,\mathbf{e}_3,\mathbf{e}_1$, an orthonormal set, so $P$ is an orthogonal matrix by the definition of an orthogonal matrix. The cyclic permutation is even, so $\det(P) = 1$ and $P$ preserves orientation. In fact $P$ is the rotation by $120^\circ$ about the axis $(1,1,1)$.
Exercise 12 - Rotation Composed with a Permutation¶
Let $R$ be the rotation about the $z$-axis from Exercise 10, and let $P$ be the permutation matrix that swaps the first two coordinates,
$$ R = \begin{bmatrix}\ \ 0\ & -1\ & \ \ 0\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & \ \ 0\ & \ \ 1\ \end{bmatrix}, \qquad P = \begin{bmatrix}\ \ 0\ & \ \ 1\ & \ \ 0\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & \ \ 0\ & \ \ 1\ \end{bmatrix}. $$
- Compute $M = PR$.
- Show that $M$ is orthogonal.
- Is $M$ a rotation or a reflection? Use $\det(M)$ to justify your answer.
Check Your Answer
$$ M = PR = \begin{bmatrix}\ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & -1\ & \ \ 0\ \\ \ \ 0\ & \ \ 0\ & \ \ 1\ \end{bmatrix}. $$ $M$ is a product of orthogonal matrices, so it is orthogonal; equivalently $M^TM = I$. Since $\det(M) = \det(P)\det(R) = (-1)(1) = -1$, the matrix $M$ is orientation reversing, so it is a **reflection** — here the reflection across the $xz$-plane, $(x,y,z)\mapsto(x,-y,z)$.
Exercise 13 - Reflection Composed with a Permutation¶
Let $H$ be the Householder reflection from Exercise 9 (so $\det(H) = -1$), and let $P$ be the cyclic permutation matrix from Exercise 11 (so $\det(P) = 1$).
- Without forming the full product, determine $\det(HP)$ and state whether $HP$ preserves or reverses orientation.
- Explain why $HP$ must be an orthogonal matrix.
Check Your Answer
By multiplicativity of the determinant, $\det(HP) = \det(H)\det(P) = (-1)(1) = -1$, so $HP$ reverses orientation. It is orthogonal because a product of orthogonal matrices is orthogonal: $(HP)^T(HP) = P^TH^THP = P^T I P = P^TP = I$.
Exercise 14 - A Permutation as a Reflection¶
Let $P$ be the permutation matrix that swaps the first and third coordinates.
- Write down $P$ and compute $\det(P)$.
- Is $P$ a rotation, a reflection, or neither? Justify your answer.
Check Your Answer
$$ P = \begin{bmatrix}\ \ 0\ & \ \ 0\ & \ \ 1\ \\ \ \ 0\ & \ \ 1\ & \ \ 0\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \end{bmatrix}, \qquad \det(P) = -1. $$ $P$ is orthogonal with $\det(P) = -1$, and it satisfies $P = P^T$ and $P^2 = I$. It fixes every vector in the plane $x = z$ (spanned by $(1,0,1)$ and $(0,1,0)$) and negates $(1,0,-1)$, so $P$ is the **reflection** across the plane $x = z$, not a rotation.
True/False Questions¶
For each statement, decide whether it is always true; a statement that is only sometimes true is false. If true, give a general argument; if false, give a counterexample with specific numbers.
- An orthonormal set of vectors in $\mathbb{R}^n$ is always a basis for $\mathbb{R}^n$.
Follow Along
**FALSE.** An orthonormal set is a basis for its span, and it spans all of $\mathbb{R}^n$ only when it contains $n$ vectors. Counterexample: in $\mathbb{R}^3$ the set $\{\mathbf{e}_1\} = \{(1,0,0)\}$ is orthonormal but contains a single vector, so it cannot span the three dimensional space $\mathbb{R}^3$.
- Orthogonal matrices preserve the norm of every vector.
Follow Along
**TRUE.** If $Q$ is orthogonal then $Q^TQ = I$, so for every vector $\mathbf{x}$, $\lVert Q\mathbf{x}\rVert^2 = (Q\mathbf{x})^T(Q\mathbf{x}) = \mathbf{x}^TQ^TQ\mathbf{x} = \mathbf{x}^T\mathbf{x} = \lVert\mathbf{x}\rVert^2$. Taking square roots gives $\lVert Q\mathbf{x}\rVert = \lVert\mathbf{x}\rVert$.
- Every rotation matrix in $\mathbb{R}^3$ has determinant equal to $1$.
Follow Along
**TRUE.** Every orthogonal matrix has $\det = \pm 1$, and a rotation is orientation preserving, which is exactly the case $\det = +1$ (the special orthogonal group $SO(3)$).
- Every permutation matrix has determinant $1$.
Follow Along
**FALSE.** The determinant of a permutation matrix equals the sign of the permutation. Counterexample: the transposition that swaps the first two coordinates,
$$ P = \begin{bmatrix}\ \ 0\ & \ \ 1\ & \ \ 0\ \\ \ \ 1\ & \ \ 0\ & \ \ 0\ \\ \ \ 0\ & \ \ 0\ & \ \ 1\ \end{bmatrix}, \qquad \det(P) = -1. $$
- If a square matrix $A$ satisfies $A^TA = I$, then the rows of $A$ form an orthonormal set.
Follow Along
**TRUE.** The condition $A^TA = I$ says $A$ is orthogonal, so $A$ is invertible with $A^{-1} = A^T$, and therefore $AA^T = I$ as well. The equation $AA^T = I$ states precisely that the rows of $A$ form an orthonormal set.
Multiple Choice Questions¶
Check the box of each statement that must be true, and leave false statements unchecked. No justification is required.
- Let $Q$ be an $n\times n$ orthogonal matrix. Check each statement that must be true.
$\square\ $ The columns of $Q$ form an orthonormal set.
$\square\ $ $Q$ is invertible with $Q^{-1} = Q^T$.
$\square\ $ $\lVert Q\mathbf{x}\rVert = \lVert\mathbf{x}\rVert$ for every vector $\mathbf{x}$.
$\square\ $ $\det(Q) = 1$.
$\square\ $ $Q^TQ = I$.
Follow Along
Check the first, second, third, and fifth boxes; leave the fourth unchecked. An orthogonal matrix has orthonormal columns, satisfies $Q^TQ = I$, is invertible with $Q^{-1} = Q^T$, and preserves norms; however its determinant is $\pm 1$, so $\det(Q) = 1$ need not hold (a reflection has $\det = -1$).
- Let $P$ be a $3\times 3$ permutation matrix. Check each statement that must be true.
$\square\ $ $P$ is an orthogonal matrix.
$\square\ $ The columns of $P$ are the standard basis vectors in some order.
$\square\ $ $\det(P) = 1$.
$\square\ $ $P^{-1} = P^T$.
$\square\ $ $P$ represents a rotation of $\mathbb{R}^3$.
Follow Along
Check the first, second, and fourth boxes; leave the third and fifth unchecked. A permutation matrix is orthogonal with columns equal to the standard basis vectors in some order, so $P^{-1} = P^T$. Its determinant is the sign of the permutation, which can be $-1$; when $\det(P) = -1$ the matrix reverses orientation and is a reflection rather than a rotation.
- Let $R = I - 2\mathbf{n}\mathbf{n}^T$ be the Householder reflection for a unit vector $\mathbf{n}\in\mathbb{R}^3$. Check each statement that must be true.
$\square\ $ $R$ is an orthogonal matrix.
$\square\ $ $\det(R) = -1$.
$\square\ $ $R\mathbf{n} = -\mathbf{n}$.
$\square\ $ $R$ is symmetric.
$\square\ $ $R$ is a rotation.
Follow Along
Check the first, second, third, and fourth boxes; leave the fifth unchecked. A Householder reflection is orthogonal and symmetric, satisfies $R\mathbf{n} = -\mathbf{n}$, and has $\det(R) = -1$. Because it reverses orientation it is a reflection, not a rotation.